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The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 44, 48-49 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the enablement requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. The disclosure fails to provide the details as how the captured image of an 
object, which is just a bitmap, can be used to select a specific component of the object 
for displaying its annotating information. It is well known to select a point on a bitmap 
image, but it is unclear in Applicant's specification as how to associate the selected 
position with a specific part of the displayed object as claimed. 

Claim 50 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and/or use the invention. It 
is unclear as how the selected element is identified for annotated on the captured image 
based on the element's position (received from the beacon) in the real space. 

Claim 51 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and/or use the invention. It 
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is unclear as how the selected element is identified on the captured image based on the 
position and shooting direction of said camera. 



The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 44, 48-49 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Rose et al. (Annotating Real-World Objects Using Augmented Reality) in view of 
Harrison et al (6,611,725). 

As per claim 44, Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 

"capturing a digital image of a view having an element with a camera" (Rose, 
page 2, video camera captures the object on scene; "currently we use a video monitor 
to represent the user's view, ..." - lines 2-3); 

"selecting an element from the captured image" (Rose, page 10, last paragraph); 

"obtaining an identification of the element" (Rose, page 9, section 6 Model 
Annotations, 2 nd paragraph); 



Application/Control Number: 09/81 7,31 4 Page 4 

Art Unit: 2628 

"relating the identification to annotating data associated with said elements; 
superimposing said annotating data on said view" (Rose, page 9, section 6 Model 
Annotations);and 

"causing the data to be displayed on a display device" (Rose, page 10, figure 

6.1). 

It is noted that Rose does not teach elements displayed with its annotating 
information is selected "from the captured image" as claimed. However, Harrison 
teaches how to generate an annotation to a display 2D object (Harrison, column 7, lines 
35-58). The decision to use a 3D pointer for selecting a part of the 3D object as in the 
Rose reference or a 2D pointer in combine with other 3D data structure for selecting a 
part of an object represented in the 2D image as in Harrison reference are both well 
known in the art. The motivation for use a 2D image to select a part for annotation is to 
take advantage of 2D monitor display. It is Rose does not teach the camera belongs to 
"a mobile phone" as claimed. However, since the mobile phone has no involvement in 
the process of capturing an image, selecting an element for annotating, finding and 
displaying the annotating data on the display, it would have been obvious to capture 
images by a camera of a mobile phone or a mobile device (e.g., a laptop) is just a 
matter of design choice. 

RESPONSE TO APPLICANT'S ARGUMENTS : 

Applicant's arguments filed May 18, 2010 have been fully considered but they 
are not deemed to be persuasive. 
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Applicant argues that Rose uses a 3D pointing device to select a component of 
the real 3D object, not a 2D pointing device to select an element of the 2D image as 
claimed. The new reference of Harrison teaches how to select an element of a 2D 
image as claimed. The argument deems to be moot due to new ground of rejection. 

Applicant argues that Rose does not teach the camera is attached to "a mobile 
phone." However, the cited references of Rose and Harrison teach all the processes of 
the method and the claimed "mobile phone" has no involvement in the claimed steps of 
the method. The portability of a mobile phone for "image taken on the go" is just a 
matter design choice in view of a mobile laptop containing the software applications as 
described in the Rose and Harrison references. 

As per claim 48, Rose teaches the claimed "system for annotating" comprising: 

"a mobile phone including a camera for capturing a digital image of a view" 
(Rose, page 2, video camera captures the object on scene; "currently we use a video 
monitor to represent the user's view, ..." - lines 2-3); 

"means for obtaining an identification of the element" (Rose, page 9, section 6 
Model Annotations, 2 nd paragraph); 

"means for relating the identification to annotating data associated with said 
elements; means for superimposing said annotating data on said view" (Rose, page 9, 
section 6 Model Annotations); and 

"a display for displaying data" (Rose, page 1 0, figure 6.1 ). 
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It is noted that Rose does not teach the elements displayed with its annotating 
information is selected "from the captured view" as claimed. However, Harrison teaches 
how to generate an annotation to a display 2D object (Harrison, column 7, lines 35-58). 
The decision to use a 3D pointer for selecting a part of the 3D object as in the Rose 
reference or a 2D pointer in combine with other 3D data structure for selecting a part of 
an object represented in the 2D image as in Ellenby reference are both well known in 
the art. The motivation for use a 2D image to select a part for annotation is to take 
advantage of 2D monitor display. It is Rose does not teach the camera belongs to "a 
mobile phone" as claimed. However, since the mobile phone has no involvement in the 
process of capturing an image, selecting an element for annotating, finding and 
displaying the annotating data on the display, it would have been obvious to capture 
images by a camera of a mobile phone or a mobile device (e.g., a laptop) is just a 
matter of design choice. 

As per claim 49, Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 

"capturing a digital image of a view having an element with a camera; displaying 
said view on a display" (Rose, page 2, video camera captures the object on scene; 
"currently we use a video monitor to represent the user's view, ..." - lines 2-3); 

"selecting an element to be annotated" (Rose, page 10, last paragraph); 



Application/Control Number: 09/817,314 Page 7 

Art Unit: 2628 

"relating the identification to annotating data associated with said elements; 
superimposing said annotating data on said view" (Rose, page 9, section 6 Model 
Annotations);and 

"super imposing said annotating data on said view" (Rose, page 10, figure 6.1). 

It is noted that Rose does not teach elements displayed with its annotating 
information is selected "by pointing on said element on said display" as claimed. 
However, Harrison teaches how to generate an annotation to a display 2D object by 
pointing on said element on said display (Harrison, column 7, lines 35-58). The 
decision to use a 3D pointer for selecting a part of the 3D object as in the Rose 
reference or a 2D pointer in combine with other 3D data structure for selecting a part of 
an object represented in the 2D image as in Ellenby reference are both well known in 
the art. The motivation for use a 2D image to select a part for annotation is to take 
advantage of 2D monitor display. 

Claims 44, 48-49, and 51 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rose et al. (Annotating Real-World Objects Using Augmented 
Reality) in view of Ellenby et al (6,307,556). 

As per claim 44, Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 
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"capturing a digital image of a view having an element with a camera" (Rose, 
page 2, video camera captures the object on scene; "currently we use a video monitor 
to represent the user's view, ..." - lines 2-3); 

"selecting an element from the captured image" (Rose, page 10, last paragraph); 

"obtaining an identification of the element" (Rose, page 9, section 6 Model 
Annotations, 2 nd paragraph); 

"relating the identification to annotating data associated with said elements; 
superimposing said annotating data on said view" (Rose, page 9, section 6 Model 
Annotations);and 

"causing the data to be displayed on a display device" (Rose, page 10, figure 

6.1). 

It is noted that Rose does not teach elements displayed with its annotating 
information is selected "from the captured image" as claimed. However, Ellenby 
teaches how to generate an annotation to a display 2D object (Ellenby, column 7, lines 
47-61 ). The decision to use a 3D pointer for selecting a part of the 3D object as in the 
Rose reference or a 2D pointer in combine with other 3D data structure for selecting a 
part of an object represented in the 2D image as in Ellenby reference are both well 
known in the art. The motivation for use a 2D image to select a part for annotation is to 
take advantage of 2D monitor display. It is Rose does not teach the camera belongs to 
"a mobile phone" as claimed. However, since the mobile phone has no involvement in 
the process of capturing an image, selecting an element for annotating, finding and 
displaying the annotating data on the display, it would have been obvious to capture 
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images by a camera of a mobile phone or a mobile device (e.g., a laptop) is just a 
matter of design choice. 

RESPONSE TO APPLICANT'S ARGUMENTS : 

Applicant's arguments filed May 18, 2010 have been fully considered but they 
are not deemed to be persuasive. 

Applicant argues that Rose uses a 3D pointing device to select a component of 
the real 3D object, not a 2D pointing device to select an element of the 2D image as 
claimed. The new reference of Ellenby teaches how to select an element of a 2D image 
as claimed. The argument deems to be moot due to new ground of rejection. 

Applicant argues that Rose does not teach the camera is attached to "a mobile 
phone." However, the cited references of Rose and Ellenby teach all the processes of 
the method and the claimed "mobile phone" has no involvement in the claimed steps of 
the method. The portability of a mobile phone for "image taken on the go" is just a 
matter design choice in view of a mobile laptop containing the software applications as 
described in the Rose and Ellenby references. 

As per claim 48, Rose teaches the claimed "system for annotating" comprising: 
"a mobile phone including a camera for capturing a digital image of a view" 

(Rose, page 2, video camera captures the object on scene; "currently we use a video 

monitor to represent the user's view, ..." - lines 2-3); 
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"means for obtaining an identification of the element" (Rose, page 9, section 6 
Model Annotations, 2 nd paragraph); 

"means for relating the identification to annotating data associated with said 
elements; means for superimposing said annotating data on said view" (Rose, page 9, 
section 6 Model Annotations); and 

"a display for displaying data" (Rose, page 1 0, figure 6.1 ). 

It is noted that Rose does not teach the elements displayed with its annotating 
information is selected "from the captured view" as claimed. However, Ellenby teaches 
how to generate an annotation to a display 2D object (Ellenby, column 7, lines 47-61). 
The decision to use a 3D pointer for selecting a part of the 3D object as in the Rose 
reference or a 2D pointer in combine with other 3D data structure for selecting a part of 
an object represented in the 2D image as in Ellenby reference are both well known in 
the art. The motivation for use a 2D image to select a part for annotation is to take 
advantage of 2D monitor display. It is Rose does not teach the camera belongs to "a 
mobile phone" as claimed. However, since the mobile phone has no involvement in the 
process of capturing an image, selecting an element for annotating, finding and 
displaying the annotating data on the display, it would have been obvious to capture 
images by a camera of a mobile phone or a mobile device (e.g., a laptop) is just a 
matter of design choice. 

As per claim 49, Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 
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"capturing a digital image of a view having an element with a camera; displaying 
said view on a display" (Rose, page 2, video camera captures the object on scene; 
"currently we use a video monitor to represent the user's view, ..." - lines 2-3); 

"selecting an element to be annotated" (Rose, page 10, last paragraph); 
"relating the identification to annotating data associated with said elements; 
superimposing said annotating data on said view" (Rose, page 9, section 6 Model 
Annotations);and 

"super imposing said annotating data on said view" (Rose, page 10, figure 6.1). 

It is noted that Rose does not teach elements displayed with its annotating 
information is selected "by pointing on said element on said display" as claimed. 
However, Ellenby teaches how to generate an annotation to a display 2D object by 
pointing on said element on said display (Ellenby, column 7, lines 47-61). The decision 
to use a 3D pointer for selecting a part of the 3D object as in the Rose reference or a 2D 
pointer in combine with other 3D data structure for selecting a part of an object 
represented in the 2D image as in Ellenby reference are both well known in the art. The 
motivation for use a 2D image to select a part for annotation is to take advantage of 2D 
monitor display. 

As per claim 51 , Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 
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"capturing a digital image of a view having an element with a camera; displaying 
said view on a display" (Rose, page 2, video camera captures the object on scene; 
"currently we use a video monitor to represent the user's view, ..." - lines 2-3); 

"relating a selected element to annotating data associated with the element" 
(Rose, page 9, section 6 Model Annotations);and 

"super imposing said annotating data on said view" (Rose, page 10, figure 6.1). 

It is noted that Rose does not teach elements displayed with its annotating 
information is selected "using said positions, said shooting direction of said camera and 
visual cues" as claimed. However, Ellenby teaches how to generate an annotation to a 
display 2D object by "using said positions, said shooting direction of said camera and 
visual cues" (Ellenby, column 4, lines 36-38, column 6, lines 56-66; column 7, lines 26- 
46). The motivation for use "said positions, said shooting direction of said camera and 
visual cues" because they are well known to accurately identify a selected element on 
the displayed scene (Ellenby, column 5, lines 35-47). 

Claim 50 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rose et 
al. (Annotating Real-World Objects Using Augmented Reality) in view of Sanderford et 
al. (5,917,449). 

As per claim 50, Rose teaches the claimed "computerized method for annotating" 
comprising the steps of: 
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"capturing a digital image of a view having an element with a camera; displaying 
said view on a display" (Rose, page 2, video camera captures the object on scene; 
"currently we use a video monitor to represent the user's view, ..." - lines 2-3); 

"determining the position of said element and for retreiving meta-information 
associated with the element" (Rose, page 9, section 6 Model Annotations);and 

"super imposing said annotating data corresponding to said meta-information on 
said view" (Rose, page 10, figure 6.1). 

It is noted that Rose does not teach the element's position and meta-information 
is received "using radio signals" as claimed. However, Sanderford teaches how to 
sending the position and information using a radio signal (Sanderford, the radio beacon 
406, column 10, lines 22-46, column 12, line 50 to column 13, line 3, lines ). The 
motivation for using radio signal to transmit the element's position and its meta- 
information is to take advantage of accurately determining the element's information 
(Sanderford, column 10, lines 22-24). 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to PHU NGUYEN whose telephone number is (571)272- 
7645. The examiner can normally be reached on M-F/8AM-4:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on (571 ) 272 7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/ Phu K. Nguyen/ 

Primary Examiner, Art Unit 2628 



